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VIHAT BEHAVIORAL OBJECTIVES DO * 
PHYSICS TESTS ACTUALLY MEASURE: PART II. 

by 

R.6.E. Wtlas, Ph.D. 
Associate Professor of Education 
Ohio University 
Athens, Ohio 



Introduction and Rationale 

Much emphasis Is given to Identification and specification of 
objectives of science teaching. Increasing attention Is focused on 
development of behavioral objectives which are pertinent to science. 
The ever Increasing list of publications on the subject at least 
demonstrates a pre-occupatlon with the subject. This may be taken 
as an Index of the significance and value which science educators and 
teachers attach to It. 

Most science teachers and educators are aware of the basic 
objectives which are generally accepted for science teaching. Those 
who work with curriculum development and materials are also continuously 
striving to provide experiences which are congruent with the. stated 
objectives. 

Students, at the high school level (as well as at the doctorate 
level), strive to achieve those ends which are used for their evaluation. 
Teachers, consciously or subconsciously, emphasize those concepts and 
objectives which happened to be dominant on an evaluatlpg-lnstrument. 
This Instrument may be a standardized achievement test or other type of 
test. Smith and Tyler stressed the effect of teachers' conception of 
achievement tests on their teaching as follows: 

Teachers, too, are frequently Influenced by their conception 
of the achievement tests used. If these tests are thought to 
emphasize certain points, these points will be emphasized In 
teaching even though they are not Included In the plan of the 
course. 

It follows, then, that objectives which teachers stress most In their 
teaching and the ones which students attempt to achieve In their learning; 
— are those objectives which are included on achievement tests. If this 
premise prevails, then It Is extremely significant for scienfee teachers,' 
scientists and educators to scrutinize, thoroughly, various science achieve 
ment and other evaluation Instruments in terms of the objectives which 
which these Instruments measure. 



*Paper presented at the Annual Convention. of the National Association 
for Research in Science Teaching, Los Angeles, Cal1fom1a»Jtorch 19,. 1975. 



2 



Probably just as much - if not n»re - attention need to be given to 
assessment and analysis of achievement tests as It Is geven to the 
Identification and statement of objectives for science teaching. Tyler 
emphasized this need early as follows: 

A satisfactory test for examination In an^ subject Is an 
Instrument which gives us evidence of tHedegree to which 
students are reaching the objectives of the subject. One 
major defect of typical examinations has been the fact that 
these examinations have given evidence with reference to only 
a limited number of objectives.... Hence, a fundamental task In 
constructing achievement tests Is to make certain that the 
Important objectives of the subject and the course are adequately 
measured. . . .Tests need to be constructed for each of the Important 
objectives of the course. 2 

More recently, Mager re-emphasized Tyler's point regarding the organic 
relationship between tests and course objectives. Nager states: 

Tests and examinations are the mile points along the road 
to learning and are supposed to tell the teacher and the 
student the degree to which both have been successful in 
their achievement of the course objectives. 3 

Focusing on science teaching, Boeck made suggestions regarding 
improvements in evaluating science teaching and achievements through 
inqiroved achievement tests. Boeck states: 

Evaluation of teaching and achievement in science may be 
characterized as being centered on measurement of acquired 
factual information. Unfortunately, this is continuing in 
spite of our knowing that this type of outcome is short- 
lived at best and of less lasting value than some of the ' 
accepted objectives of science instruction Irrespective of 
future plans of the pupils. Examination techniques and 
tests to measure achievement in these objectives, including 
the ability to use the methods of science, do reflective 
thinking, use problem solving techniques, and the degree 
to which scientific attitudes are developed and enq)loyed by 
pupils: need development, validation and standardization.^ 

The Committee on Rethinking Science Education recognized the need 
for establishing evaluation in terms of the stated objectives of 
science teaching. The Connlttee stressed the functional relationship 
between the two components as follows: 
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Objectives Indicate the nature of the educational 
endeavor and denote the direction It should take... 
Evaluation should be in terms of all the goals that 
are set for the teaching process .TT'" 

The above review demonstrates that a functional relationship need 
to be established between all, the accepted goals of teaching science 
and the Instruments which are used for evaluation of science teaching. 
It Is the main thesis of this paper that no matter how elaborately 
or extensively we state objectives or how much we value them, such 
objectives will receive little. If any, attention from science teachers 
and students unless these objectives are Included clearly and Intentionally 
In the evaluation instruemtn. In most cases thesef evaluation Instruments 
happen to be standardized achievement tests. 



The Problem 

Few studies focused attention on what physics tests actually 
measure in terms of the generally accepted objectives for science 
teaching. In 1949 Burke reported on a study of critical thinking In 
physics as measured by physics tests. Burke concluded his findings 
on a discouraging note: 

..-.tests for critical thinking are needed. Unfortunately, 
there Is lack of experience In the field of testing for 
critical thinking, and one should not hope for too rapid 
progress." 

In 1971, Hitlas reported a study on various objectives for science 
teaobipg which were measured by a sample of standardized physics 
achievement tests. The tests used were mostly constructed in the late 
1950's and early 1960's. Hitlas' findings showed that the tests primarily 
emphasized objectives such as recall of factual knowledge, recall of 
physics principles, and Interpretation and mathematical application of 
these principles. Very little attention was given - In most of the tests - 
to evaluation of other objectives such as how to draw conclusions from 
given data» how to Interpret graphs, how to test hypotheses, or how to 
assess attitudes toward superstitions or suspending judgment.^ 

The Purpose of this Study: 

The present study Is a follow-up on on the earlier one done by 
the author (Nitlas, 1971). The primary purpose of this Investigation was 
to Identify the objectives, for science teaching, which are measured by 
a sample of the most current editions of four standardized physics 
achievement tests. 
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More specifically, this study attempted to ansvfer the following 
questions: 

1. What behavioral/specific objectives of science teaching does 
a sample of current standardized physics achievement tests 
measure? 

2. How do these tests compare with one another with regard to the 
emphasis they give, to an Identified list of science teaching 
objectives? 

3. How does this sample of current standardized ph./s1cs 
achievement tests compare, generally, with the findings of the 
earlier study, reported by the author, regarding- measurement 
of various objectives of science teaching?. ^ 



Methodology . 
I. Development of the Instrument: 

a. A thorough study of the literature was conducted to identify the 
major objectives of science teaching which could be measured 
through a paper-and-pencil type test. Eleven general objectives 
were identified and selected on the basis of their frequency 

and consistent appearance in the literature oh science education? 

b. The eleven general objectives identified in 1 above were then 
stated in specific behavioral terms. Literature related to this 
area was thoroughly explov^d with attention given primarily to 
specific behavior related to physics** 

c. The following "check-list" was then developed as the Instrument 
for the analysis of the tests. This Instrument is composed of 
all eleven general objectives and their sub categories - which 
are stated in specific/behavioral termst** 
These objectives follow: 



1 . Knowledge of Factual Information: 

The solution of the .test Items requires the student to: 

1.1 Recall specific facts of physics 

1.2 Recall general physical principles 

1.3 Define common technical terms used ^n physics texts and publications 

1.4 Recall certain historic facts or events in physics 



* See selected bibliographic material related to this area. 

** See selected bibliographic material related to this area; 

*** This Instrument was used in an earlier stucly reported by the author 
(Mitias, 1971). 
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2. Application of Physics Principles: 

The solution of the test items requires the student tot 

2.1 Interpret or explain physics principles which have been given 

2.2 Interpret or explain physics principles which the student recalls 
Explanation was used to mean, also, solution of physics- 
mathematical problems which required use of physical principles. 

3. Ability to Identify Problems; 

The solution of the test itene requires the student to: 

3.1 Identify problematic situations 

3.2 Differentiate between facts and problems 

3.3 Differentiate between problems and hypotheses 

3.4 Isolate single major Idea of a problem 

3.5 State a problem 

3.6 Evaluate problems in terms of personal and social needs. 

4. Ability to Analyze Problems ; 

The solution of the test items requires the student to: 

4.1 Pick out and/or define key words as a means of better understanding 
- of the problem( situation) 

4.2 Identify the relatlonshlp(s) of minor problems to the major one 

4.3 Identify the basic assumptions of the problem. 

5. Ability to Collect Information: 

The solution of the test Items requires the student to: 

5.1 Differentiate authoritative and non-authorltlatlve (reliable 
and less reliable) sources of Information 

5.2 Criticize articles, editorials, etc. 

5.3 Select data which are pertinent to a problem 

5.4 Arrange and/or make graphs of data 

6. Ability to Test Hypotheses: 

The solution of the test items requires the student to: 

6.1 Select an hypothesis, from a list of hypotheses, which 
most adequately explains alven data 

6.2 Determine the accordance (agreement) of an hypothesis with 
the data 

6.3 Determine the adequacy of an hypothesis In explaining the 
problem 

6.4 Recognize the tentative nature of an hypothesis. 

7. Ability to Interpret Graphs and Data : 

The solution of the test items requires the student to: 

7.1 Obtain specific Information from graphical material 

7.2 State relationships as tentative generalizations which may 
serve as hypotheses. 

7.3 Criticize Inferences drawn from data by recognizing whether 
an inference is an implication of the data, unrelated to the 
data, or contradicted the data 

7.4 Formulate the main Ideas presented in the data 
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8. Attitude Toward Superstition : 

The solution of the test items requires the student to: 

8.1 Assess belief or non-belief in charms or signs of good or bad luck 

8.2 Identify statements that are not definitions, are not verifiable 
by observations or do not have implications which are verifiable 
by observations, and are not mathematical or logical propositions 

8.3 Interpret "inysterious" and 'strange" situations by natural causes 

9. Attitude to Suspend Judgment ; 

The solution of the test Items requires the student to: 

9.1 Criticize faulty deductive reasoning 

9.2 Delay taking a decision or passing judgment if data were insuffi- 
cient or inadequate 

9.3 Identify exaggerations in judgment or conclusions based on given 
data 

9.4 Demand observational evidente or other pertinent data before 
accepting statements as facts 

10. Ability to Draw Conclusions : 

The solution of the test items requires the student to: 

10.1 Accumulate and relate tested evidence which supports a conclusion 

10.2 Evaluate conclusions in relationship to assumptions set up 
for the problem 

10.3 Apply conclusions to new situations 

11. Ability to Differentiate Between Various Components in Thought Processes : 
The solution of the test items requires the student to: 

11.1 Distinguish between an hypothesis and a conclusion 

11.2 Differentiate between a conclusion and a principle 

11.3 Isolate a fact from a broader principle 



II. Selection of the Physics Achievement Tests: 

Four high school standardized physics achievement tests were selected 
for this study. Selection was based on the following: 

a. All are paper-and-pencil type tests 

b. All are widely used 

.. c. All publishers granted permission for use in the study 
d. All are latest editions of the tests used in an earlier study. 

The four tests included in this study are: 

1. Dunninq-Abeles Physics Test , Form E, Harcourt, Brace & World, 
Inc., New York, 1967. 

2. Tests of the Physical Science Study Committee , Form F, 
Educational Vesting Service, Princeton, N.J., 1967. 
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3. Every Pupil Scholarship Test, Bureau of Educational Measurement 
Kansas State Teachers College, Emporia, Kansas, 1970. 

4. Minnesota High School Achievement Examination, Form EH(Rev.), • 
Test No. 17, American Guidance service. Circle Pines, 
Minnesota, 1988. 

This last test is the newest edition of the one used In the earlier 
study under the title: Midwest High School Achievement Examination. 
Form G, Educational Publishers, Inc., 1960. '■ 



III'. Analysis of Test Items: 

Each item on each of the four tests was analyzed in terms of the 
"check-list" developed in I. above. This process of analysis attempted 
to identify the minimum behavioral processes needed by a hypothetical 
high school student to answer the test item correctly.^ The analysis^was 
done in the following format: 

Test item general objective(s) specific/behavioral objective 

1 1,2 1.1, 1.3, 2.2 



IV. Organization of the Data: 

a. The specific/behavioral objectives which were -involved in the 
answer of each test Item were identified. The general objectives 
were then listed whenever at least one specific/behavioral 
objective of that general objective was included in the solution. 
This was done for each test item on each test. 

b. The frequencies of each behavioral objective and of each general 
objective were calculated on each test. The percentages were 
also calculated in relationship to the total number of Items on 
each of the four tests. 

c. Results of the analysis were put in tables regarding each general 
objective and its behavioral components. A table was made to 
compare the frequency and percentage of each of the behavioral 
objectives on each of the four tests. This was also done to 
compare each of the present four tests with its parallel 
earlier edition. For econoiqy's saf s, however, the results are 
included here in two surnmary tables. Table I presents the 
findings of this study, and answers questions i and 2 in the 
problem statement. Table II presents the findingis of the earlier 
study. Comparison between tables I and II would answer question 
3 in the statement of the problem. 
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Findings 

Findings of this study are presented, in a condensed manner, in Table 
I. Table II is presented here, from an earlier study* for comparative 
purposes as indicated above. The major findings of this study may be 
sunmarized as follows: 

1. General objective #1: Knowledge of factual information (physics) 
and its behavioral components (1.1, 1.2, 1.3) are heavily 
emphasized by all four physics achievement tests, as Table I 
indicates. Behavioral component 1.4 : recall certain historic 
facts or events in physics, however, seems to receive no 
attention in any of these tests. The results of the present 
study with respect to general objective #1 seem to be 
consistent with the earlier study regarding these findings 
when comparing Tables I and II. 

^ 2. General objective #2: Application of physics principles and 
its behavioral component 2.2 seem to rank second in terms of 
what these tests emphasized. There is much less attention given 
to behavioral component 2.1 (interpret or explain physics 
principles which have been given). The emphasis seems to focus 
on interpretation and/or solution of physics problems which 
require recall of general physics principles (2.2) as Table I 
indicates. Comparing Tables I and II shows an apparent 
increase in the emphasis given to this objective in the current 
edition of the tests as compared to the earlier edition. It 
seems that all four tests have increased remarkably in this 
general objective. The PSSC test seems to have increased the 
most followed closely by the Dunning-Abeles test. 

3. General Objective #3: Ability to identify problems and its 
behavioral component objective 3.4 was emphasized only by the 
PSSC test, and to a lesser degree by the Minnesota Achievement 
test. Behavioral components 3.2, 3.3, 3.5 and 3.6 did not 
seem to be measured by any of the tests. The results here seem 
to agree with those of the earlier stydy. The major exception 
seems to be the degree to which the PSSC test emphasize 
behavioral component 3.4 (Isolate single major idea of a 
problem) which increased from 26.9% to 53.7J; in the present 
edition. 

4. General objective #4: Ability to analyze problems, and its 
behavioral components seemed to receive little attention from 
these tests with the exception of the PSSC test. Tables I and 

II show that this general objective increased from 21.8% to 57. 5X 
in the present study. One, however, must point out that this 
increase focused mainly on behavioral objective 4.2 to the 
exclusion of other behavioral components. Other tests seem to 
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have maintained about the same level of attention given to this 
objective or even declined slightly. 

5. General objective #5: Ability to collect Information and Its 
behavioral components 5.1* 5.2, 5.3, and 5.4 were not measured 
by any of the tests used In this study as Table I Indicates. 
CDfliparlson between Tables I and II show that whatever little the 
earlier tests have measured seemed to have been eliminated 

In the more current editions. 

6. General objective #6: Ability to test h^'potheses and Its 
behavioral components 6.1 and 6.2 appeared to be measured to 
a small degree (10%) only by the PSSC test. The rest of the 
tests Ignored It completely. Comparing Tables I and 11 
show that the PSSC test Improved slightly on measuring this 
objective while the other tests seem to have abandoned any 
attempt toward such measurement. 

7. General objective #7: Ability to Interpret graphs and data, and 
Its components 7.1 and 7.4 were emphasized much more In this 
study by the tests with the exception of Minnesota High School 
Achievement Examination. Comparing Tables I and 11 shows PSSC 
with 70X of the Items requiring this skill corapai*ed to 19.2X 

In the earlier edition and Dunnlng-Abeles with 36. IX of the 
Items requiring some skill In this area compared to 0. OX In 
the earlier edition. 

8. Examining Table I (as well as Table II) we see that, four 
general objectives 8-11 and their behavioral components as 
listed on the Instrument were not measured by any of the tests 
used. This seems to hold true on the earlier study as well 

as the present one. The general objectives which were 
neglected on these tests are: Attitude toward superstition. 
Attitude to suspend judgment. Ability to draw conclusions, 
and Ability to differentiate between various components In 
thought processes. 



General Comments 

This study has attempted to Identify behavioral objectives which 
current editions of four standardized achievement tests measure^ It 
also attempted to compare the findings with those of a previous study 
of earlier editions of these tests In order to Identify any changies In 
the emphases which these tests place on various science teaching 
objectives. 

The findings of this study pointed to several changes. Among these 
were the attention given, at least by some of these tests, to (a) the 
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application of general physics principles in terms of explanation, 
interpretation and solution of related mathematical problems* (b) 
the interpretation of graphs and data and formulating main Ideas 
presented In the graphical data. 

Major attention of the tests seems to focus almost In every 
item on knowledge of factual information in physics. This Included 
recall of facts, general physical principles and definition of technical 
physics terms. 

Very little or no attention was given to several objectives 
of science teaching. Among these were the following general objectives: 

Ability to collect information, ability to test hypotheses, 
attitude toward superstition, attitude to suspend judgment,. ability to 
draw conclusions, and the ability to differentiate between various 
components in thought processes (1e. hypotheses, conclusions, facts, 
and principles). 



Conclusion 

It is encouraging to observe a change in some physics achievement 
tests toward more emphasis on other objectives beside that of recall of 
factual Information. However, this change seems to be less than 
universal. In addition, one still observes the lack of evaluation of 
science teaching objectives which have been considered vitally 
important among science educators. This seems to hold true for physics 
tests developed in the last twenty years. While changes In curricula, 
methodology, etc. seem to have dominated these twenty years, evaluation 
instruments seem to trail far behind. If the earlier premise of this 
study that individuals, including students, strive to achieve the ends 
which are used as criteria for their evaluation (often teacher made tests 
are patterned after standardized Instruments) holds true, then one 
expects high school students to give greatest attention to the objectives 
measured by these achievement tests. This would happen no matter what 
other objectives one may state for the curriculum. Thus, it becomes 
imperative that in order to achieve other vital objectives for 'science 
teaching that these objectives be Included on evaluation Instruments; 
In this case standardized achievement tests. Otherwise, results of 
curriculum and methodological changes would be Incidental and students' 
achievement along these objectives would be very difficult to assess. 
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